Using Psychometric Technology in Educational Assessment: The Case of a Schema-Based Isomorphic Approach to the Automatic Generation of Quantitative Reasoning Items

Autor/inn/en	Arendasy, Martin; Sommer, Markus
Titel	Using Psychometric Technology in Educational Assessment: The Case of a Schema-Based Isomorphic Approach to the Automatic Generation of Quantitative Reasoning Items
Quelle	In: Learning and Individual Differences, 17 (2007) 4, S.366-383 (18 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1041-6080
DOI	10.1016/j.lindif.2007.03.005
Schlagwörter	Schemata (Cognition); Test Items; Intelligent Tutoring Systems; Construct Validity; Quality Control; Problem Solving; Program Effectiveness; Factor Analysis; Psychometrics; Algebra; Word Problems (Mathematics); Educational Assessment + Suchen Sie Ihr Suchwort? Cognition; Schema; Kognition; Test content; Testaufgabe; Intelligentes Tutorsystem; Qualitätskontrolle; Problemlösen; Faktorenanalyse; Psychometry; Psychometrie; Textaufgabe; Education; assessment; Bewertungssystem
Abstract	This article deals with the investigation of the psychometric quality and constructs validity of algebra word problems generated by means of a schema-based version of the automatic min-max approach. Based on review of the research literature in algebra word problem solving and automatic item generation this new approach is introduced as a theory-based top-down method of automatic item generation featuring a quality control framework aimed to minimize the construct unrelated variance in the item parameters. The first study deals with the evaluation of an initial set of items. The results are replicated in the second study using a larger item set which also allows the investigation of the construct representation of the generated item. Since construct unrelated variance components (e.g. reading comprehension) have been controlled for in the item generation phase the results revealed some interesting insights into the cognitive processes of the actual mathematization phase of algebra word problem solving. The third study investigated the nomothetic span is using hierarchical confirmatory factor analysis. The results argue for the convergent and discriminant validity of the automatically generated items. Taken together, the results indicate that the automatic generation of construct valid algebra word problems at a high psychometric level is viable. The discussion is thus concerned with the implications of this new approach to item generation for theory development and evaluation as well as practical benefits for educational assessment and the development of intelligent tutoring systems. (Author).
Anmerkungen	Elsevier. 6277 Sea Harbor Drive, Orlando, FL 32887-4800. Tel: 877-839-7126; Tel: 407-345-4020; Fax: 407-363-1354; e-mail: usjcs@elsevier.com; Web site: http://www.elsevier.com
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2017/4/10